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A  OQNPARISoN  Or  LINiAR  PRuGhAIMING  AND  DYNAMIC  PRUHUIMI9G 

^This  paper  considers  the  applications  and  Interrelations  of  llmear 
end  dynamic  programming  and  attempts  to  place  each  In  a  proper  perspective 
so  that  efficient  use  can  be  made  of  the  two  techniques.  ^ 

61.  The  Philosophies 


Linear  programming  adopts  an  intentionally  simple  model.  The 
complexity  of  economic  formulations  during  the  past  hundred  years 
has  far  outstripped  the  mathematician's  ability  to  solve  such 
problems  computationally.  Linear  programing  represent e  aa 'attempt 
to  reverse  this  trend  and  to  view  economic  processes  as  inter¬ 
related,  but  fundamentally  simple,  activities.  The  optimisation 
of  these  proceesee  then  become*  me  thematically  feasible. 

Dynamic  programing  concerns  itsslf  with  a  claas  of 
functional  rslatlona  that  arlss  from  multi-stags  dsclsion  processes 
possessing  certain  definite  structural  characteristics.  The 
characteristic  properties  are  axploited  to  effect  a  reduction  in 
the  dimensionality  of  the  mathematical  problem,  thereby  making  eamo 
complax  processes  amenable  to  analytic  or  computational  techniques. 

$2,  The  Models 

The  formulation  of  a  problem  in  mathematical  tens  is  the 
initial  step  towards  its  solution.  In  this,  the  model  building 
stage,  care  must  be  taken  to  aeeure  two  conditional 
1)  Ths  modal  must  b#  an  adequats  description  of  reality  and 
1)  the  model,  if  numerical  results  are  required,  mutt,  be  amenable 
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to  computational  technicues . 

As  will  be  discussed  in  cart  Bust  be  taken  that  condition 
om  is  fulfilled  when  co«.c<  dering  a  linear  programing  formulation. 

At  nottd  in  £5*  computational  feasibility  represents  the  most  important 
consideration  whan  a  dynamic  programing  modal  it  contemplatad . 

Tht  linear  programming  model  always  aetumet  the  following  fona: 


Minimi set 
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A  comon  economic  interpretation  of  thate  equations  retwitt 
from  considering  the  x  ^  to  be  activity  levels ,  the  a^  to  be  the  input 
of  the  ith  commodity  into  the  J**1  actirity  and  the  b^  to  be  upper 


limits  on  requirement  a  and  availability  of  cemoditiee.  Then  the 
model  states  t  hat  wo  with  to  carry  on  n  productive  activities,  each 
using  the  same  m  limited  resources  in  such  a  way  at  to  minimi te 
total  cost  or  maximiao  profits,  [la] 

The  typical  dynamic  programing  formulation  appears  at 


«■ 

f  (3)»  Max  3(3, P)  ♦  f  (S'(P)) 
V  P  M-l 


(3) 


This  is  a  rtcurrenct  ralation  and  it  therefore  particularly  salted 
to  processes  that  occur  over  a  sequence  of  time  period#  or  stage# , 
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k  multi-stage  process  that  is  eaeily  formulated  in  these  terns  is  an 
allocation  problem  where,  at  each  stage,  resources  are  reinvested  in 
such  a  way  as  to  maximise  total  pajroff  over  the  length  of  the  process. 

For  example,  suppose  we  are  given  a  quantity  x  of  a  resource  that  aa y  be 
divided  into  two  non-negative  parts  y  and  x-y,  During  stage  1  we  obtain 
frosi  y  a  return  g(y),  at  the  expense  of  reducing  y  to  ay  where  0  <  a  <  1; 
from  (x-y)  we  obtain  a  return  h(x-y)  at  the  expense  of  reducing  (x-y) 
to  b(x-y)  where  0  <  b  <  1.  The  process  is  now  repeated  with  the  new 
initial  quantity  ay  ♦  b(x-y),  and  so  on  for  N  periods.  How  can  one 
allocate  at  each  stage  so  as  to  maximize  the  total  return  obtained  over 
the  entire  process?  The  functional  equation  describing  this  particular 
process  is 


*3- 


■ere  f^x)  is  defined  ae  the  return  from  an  N-stage  process,  starting 
with  resource  x,  and  using  an  optimal  policy.  g(y)*h(x-y)  is  the  return 
fras  an  allocation  of  resources  into  parts  y  and  (x-y)  at  stage  1. 
a yvb(x-y)  represents  the  remaining  resources  and  is  the  Initial  condition 
for  the  (N-l)  stage  process.  The  equation  states  that  the  return  from 
an  N-slage  process  is  the  sum  of  the  return  from  stage  1  plus  the  return 
from  the  remaining  (N-l)  stages  w  here  the  initial  division  of  resources 
is  chosen  so  as  to  maximize  the  sum.  J  2 

Nomenclature 


In  view  of  the  above  examples,  it  is  easy  to  appreciate  the 
choice  of  nomenclature.  Programming,  of  course,  means  allocation 


In  each  case.  In  the  linear  programming  model  limited  resources  are 
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allocated  to  various  activities.  In  dynamic  programing  re  so  ureas 
art  alloc  a  tad  at  sach  of  several  time  periods. 

The  tern  "liaear"  emphasises  that  the  applications  concern 
situations  where  the  inputs  and  outputs  of  various  activities  can  be 
assumed  to  be  proportional  to  the  level  of  the  activity* 

Dynamic  programing  takes  its  naas  froa  the  fact  that  the 
functional  aquation  and  its  associated  computational  techniques  are 
derived  froa,  and  adapted  to,  a  process  changing  over  a  discrete  or 
continuous  tins  interval. 

Two  exceptions  should  be  aoted.  It  is  possible  to  expand  the 
static  picture  described  in  the  discussion  of  the  linear  prograaaing 

I 

aodsl  to  include  several  tiae  periods  by  the  addition  of  a  new  set  of 
restrictions  reflecting  constraints  felt  at  each  tiae  interval.  It  it 
important  to  note,  however,  that  even  where  a  dynamic  situation  is  being 
considered,  the  entire  process  is  described  by  one  set  of  equations 
sad  solved  at  one  large  problem.  .3]  In  a  dynasdc  programming  formu¬ 
lation  an  iteration  of  the  functional  eouation  corresponds  to  each 
time  interval,  ao  that  a  longer  taretlon  of  the  process  only  entails 
additional  iterations .  Hence  doubling  the  length  of  the  process  merely 

requires  twice  the  computation  time  but  ne  additienal  fermcuatlonal 
considerations.  Secondly,  dynamic  programming  does  not  necessarily 
Involve  processes  changing  over  time.  It  is  s  multi-stage  technique, 
but  often  the  stages  are  artificially  introduced  bj  considering  the 
component  activities  individually  although  they  actually  occur 
simultaneously  in  time.  For  an  example,  see  ref.  f/Tl 


The  Model  as  4  Dsecrlptlon  of  Reality 
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Linear  programing,  representing  a  simple  model,  Is  necessarily 
restricted  as  to  the  generality  of  problems  that  can  be  attacked. 
However,  if  applicable,  it  has  associated  with  it  a  very  powerful 
computational  device,  the  simplex  algorithm,  which  can  efficiently 
solve  large  systems  containing  hundreds  of  equations.  Dynamic 
progressing,  on  the  other  hand,  is  an  extremely  general  technique 
of  formulation,  but  carries  with  it  a  much  acre  limited  coaputational 
scheme.  This  section  will  discuss  three  aspects  of  problems  which  aa j 
aake  a  linear  programming  model  not  applicable  but  which  are  handled 
with  some  ease  by  dynamic  programming. 

The  linearity  assumption  is  somet  ime*  Invalid  when  applied  to 
industrial  procss  see.  The  deviation  from  linearity,  and  the  degree 
ef  exactness  required  from  the  model,  then  determine  the  applicability 
of  linear  programming.  A  problem  involving  set -up  costs  is  a  repre¬ 
sentative  member  of  the  class  of  non-linear  processes.  Here  the 

A 

situation  is  such  that  the  cost  of  carrying  on  an  activity  at  zero 
level  is  essentially  zero.  However  the  use  of  the  activity  at  even 
a  small  level  incure  a  large  cost  associated  perhaps  with  the  retool¬ 
ing  of  a  production  line  or  administrative  costs  of  placing  an  order. 
Once  this  penalty  has  been  assessed  the  assumption  of  proportional 
costs  may  become  valid.  Turning  now  to  the  functional  equation  (3) 
of  dynamic  programming,  the  policy,  P,  may  be  thought  of  as  the 
activity  level.  When  fM(S)  is  computed,  the  policy  space  is  searched 
for  a  maximizing  policy  P.  If  the  return  or  cost  function  R(S,P) 
is  given  in  tabular  form  a  discontinuity  at  the  origin  represents 
no  difficulty.  If  R(S,P)  is  a  function  applicabls  everywhere  except 
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&t  tve  origin,  the  computer  cod*  aust  include  a  test  for  P  •  0  a ad 
the  substitution  of  a  co«t  of  zsro  for  tbo  functional  cost  In  that 
rpocial  case. 

4  Aocond  consideration,  discreteness  of  tho  Aolution,  tends  to 
complicate  tbo  1  inoar  programing  formulation.  Soom  problems  of  a 
aumber  theoretic  naturo  roquiro  that  tho  activities  assume  only  integral 
value*.  For  example,  in  *  cargo  loading  problm  only  tbo  integral 
numbers  of  each  item  nay  bo  loaded  and  tbo  optimal  fitting  together  of 
tho  cargo  ovbjoct  to  a  weight  or  space  restrictions  then  boron* a  a 
discrete  number  theory  problem.  This  sort  of  a  restriction  is  nicely 
suited  to  dynamic  programming  uaing  a  digital  computer,  for  only  integral 
members  of  the  policy  domain  P  need  be  considered  and  the  function  f^S) 
need  only  be  computed  for  fixed  discrete  values. 

Finally,  there  is  a  large  family  of  problems  where  the  return 
associated  with  an  activity  le  known  only  aa  a  stochastic  function 
of  the  activity  level.  Here  again,  except  in  special  case*,  linear 
programing  techniques  are  not  applicable. 

Computational  Solution  of  the  Hodol 

Just  as  ths  previous  section  wee  devoted  almost  entirely  to  areas 
where  linear  programing  may  not  be  applicable,  this  section  will  concern 

i 

ltoelf  primarily  with  the  computational  pitfalls  of  dynamic  programming. 

And  Just  as  it  was  Asssrted  in  the  last  section  that  almost  any  conceivable 
process  sen  be  formulated  in  terms  of  dynamic  programing,  it  should  now 
be  understood  that  by  moons  of  the  simplex  and  associated  algeritlm 
*  linear  programming  problem  even  of  formidable  else  can  be  automatically 
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subduad  and  tha  solutions  axhibitsd.  5J  .  This  should  b#  notsd, 
sines ,  ths  sntirslj  succsssful  aspsets  of  sach  tachniqua  bslng  llttls 
discussad,  ons  might  arrlws  at  ths  mistaksn  corifclusion  that  asithsr 
tachniqua  offsred  much  sf  practical  Talus.  Ons  asad  only  rsfsr  to  ths 
wall  known  contributions  of  ths  sldsr  and  bsttsr  understood  linsar  pro- 
gr  inaunr  approach  or  to  ssetion  7  to  sse  ths  aids  applicability  of  both 
tschniques. 

Lot  us  rsconsldsr  ths  rscurrsnce  rslation 

Vx)  “0  M4Jt  |c(7)4h(*-/)^^1(*yb<*‘-y))]  (5) 

Mtmsrical  solution  is  obtainsd  by  first  obaarring  that  ths  rat  urn 
froai  a  1-stags  procsss  is  simply  ths  islum  of  |(y)*h(x-y)  sines  f  , 
ths  O-stags  rsturn,  is  idsntically  zsro.  A  tabls  of  f^(s)  is  computsd 
for  all  Taluss  of  %  frost  0  to  x.  Ws  ars  rsslly  saying  that  although 
ws  don*t  Know  what  our  rssourcss  will  bs  whan  ws  find  that  ths  procsss 
has  only  ons  stags  Is  ft  (this  dspsnding  on  our  aj  yst  undatsmiusd 
policy  during  ths  first  (N-l)  stags*),  for  any  initial  rssourcs,  s, 
our  final  allocation  and  ths  rsturn  frovt  it  ars  known.  Ones  T(s)  for 
all  posstols  s  is  oalculatsd,  f  (z)  can  bs  oomputsd  using  aquation  (5) 
which,  for  N  ■  2,  rslatss  f^  to  f^.  By  working  backwards  thru  ths 
ssqusncs  of  functions  f^(s)  ws  finally  arrivs  at  a  tabls  of  fa(s), 

0  <  t  <r  xt  whsrs  f  (x)  is  ths  dssired  solution,  ths  N-stags  rsturn 
snploylng  an  optimal  policy.  Hanes  for  an  M-stags  procsss  tbs  1-stags 
computation  1*  tasrsly  psrforasd  N  tiaas,  which  prsssnts  no  dlfflcuitiss 
for  ths  digital  coaputsr  progra mmr. 


One#  the  number  of  time  intervals  has  boon  relegated  to  Ho 
proper  place,  the  problem  it  reduced  to  its  true  dleenslone.  Its 
true  dimension  it  precisely  the  quantity  ef  info  met  ion  needed  te 
deeoribe  completely  the  tituttien  tt  Mgr  one  particular  stage.  In 
the  abort  sza^ple,  z,  the  quantity  of  reeourot  available,  deflate 
the  etate  of  the  system  at  any  stage  of  the  proceet.  Since  z  it  a 
tealar  number,  the  problem  it  tamed  one  dimensional  and  presents  no 
computational  difficult  let . 

However,  it  la  eaty  for  even  thle  reduced  dimensionality  te 
become  large.  In  the  Job-ehop  scheduling  problem,  ditoutaed  in  |6, 
for  ezaaple,  the  dlrntnt locality  it  equal  to  the  mum her  of  aachinee 
la  the  a  hop,  aince  one  must  know  the  at  ate  of  each  aachiae  in  order 
to  under* tend  the  etate  of  the  eyataa.  To  achieve  a  eelutloa  a  table 

i 

ef  the  function  f^(S)  must  be  computed  for  all  possible  states  3  and 
stored  for  use  la  computing  f^(3).  Where  3  is,  aey,  10  dlaensleual 
and  can  aaauma  100  values  in  each  dimeoeijoti,  the  tabular  storage  ef 
f^(3)  would  require  100^  memory  cella,  far  beyond  the  range  ef 
present  generatlen  oomputare.  So  the  dimensionality  of  the  proeeee 
must  be  about  4  or  lees  for  dynamic  programing  to  be  applicable 
in  a  purely  computational  Banner.  The  actual  performance  ef  the 
aerial  eat  ion  presents  further  interesting  difficulties,  particularly 
where  it  ie  to  be  performed  over  a  2  dr  3  dimensional  region, 
fortunately  aaqy  apparently  eoaplez  processes  can  be  reduced  to  one 
or  two  dimensional  problems  and  solved  by  dynamic  progressing  while 
in  tome  higher  dimensional  problems,  the  structure  of  the  optimal 
policy  my  be  deduced  analytically,  thus  simplifying  the  computation. 


Fonmlatlon  of  a  3— pla  ProbI 


Tna  following  la  *  variation  of  a  problaa  dltcuaaad  by  fUrxuvltg 
and  Nano*  [VJ  .  Conaidar  a  aaall  acala  job  shop*  Thorn  am  6  aachinet 
or  ratourcaa.  Twnnty-onn  dlffarant  kind*  of  ltaaa  can  ba  aanufactumd 
uaing  tht  aachinna.  Lat  x^  ba  tha  laval  of  tha  Jth  activity,  i.n.,  tha 
auabnr  of  itama  of  typa  J  to  ba  producad.  a.  .  ia  tha  aaount  of  tiaa 
on  tha  ith  aachine  raquimd  to  produca  1  of  tha  J*’*1  itaa*  ia  tha 
raturn  from  ton  unit  laval  of  tha  j*'*1  activity,  b^  ia  tha  total 
tiaa  availabla  on  tha  ith  aachina.  Furthnr,  lat  ua  aaauaa  that  it 
aakaa  no  aanaa  to  produca  a  fractional  nuabar  of  itaaa.  Finally,  va 
aaaart  that  tha  purpoaa  of  tha  procaaa  ia  to  aaxiaixa  tha  total  raturn 
by  ehooaing  an  optiaal  nuabar  of  aach  itaa  to  produca. 

Viawad  flrat  on  a  linaar  programing  modal  ua  ha  vat 


21 

jXj  Tthn  raturn) 


£• 


aubjact  to  tha  raatrictiona 


a  x  <  b 
U  J  *  i 


for  i  •  1,  2,  3,  4,  5,  6 


whara  tha  activity  lavela  x  auat  ba  non- negative  and  integral. 


At  dynamic  programing  our  aodal  appaara* 


^21  ^  ^ » ^2  *  *  * 


•  »b/)  •  Max 


^Yl+Wbl’*nIl#b2  ’•21Xl»,,*,bS“ 


*  an  intagar 
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The  process  lnrolrlng  21  items  has  been  expressed  in  terns  of  the 
first  item  end  e  process  involving  the  remaining  20  items. 

* 

As  discussed  in  the  text,  the  linear  progressing  solution  is 
made  difficult  by  the  integral  restriction,  whereas  dynamic  pro¬ 
gressing  is  obstructed  by  the  6-dimensionality  of  the  process. 

Seme  Applications 

Contained  in  this  section  will  be  found  the  statement  of 

sereral  problmu  that  hare  been  solred  by  either  linear  programming 

* 

or  dynamic  programming.  References  are  made  to  papers  containing 
more  detailed  discussions  of  the  solutions.  This  section  is 
included  for  two  purposes:  to  demonetrete  the  cethollc  nature  of 
both  techniques  end  to  eld  the  reader  in  decldli^  which  approach, 

9 

if  either,  might  prows  rewarding  if  applied  to  problems  he  may  be 
facing. 

Soma  problems  solred  by  linear  programming. 

Transportation  Problem: 

There  are  I  sources  of  an  item  and  N  sinks.  The  require¬ 
ment  of  etch  sink  is  known,  and  the  coat  of  shipment  between 
each  sourea  and  slide  is  given.  A  shipment  policy  is  deal  red 
to  meet  the  N  u emends  at  minimum  shipmeml  east,  [l^ 

Assignment  Problem: 

N  employees  ere  to  be  assigned  to  M  distinct  Jobe.  The 
aptitude  of  each  employee  for  each  Job  is  known.  Which  employee 
is  assigned  to  which  Job  in  order  to  maximise  the  total  aptitude 
of  all  smployeea  for  the  Jobs  thsy  are  filling? 
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Pstrolsum  Blinding  Problsmt 

Crude  oil  produces  guts,  straight-run  gasoline,  3  weights 
of  distillate  and  residue.  Tbeee  products  may  be  used  for  fbel 
oil  blending  or  in  several  other  processes.  The  value  of  the 

total  output  of  the  interrelated  procese  is  to  be  maximised.  f*8 

1 

Traveling  Salesman  Problem; 

To  determine  a  path  of  minimus  length  touching  49  principal 
cities  of  the  U.  3.  [?] 


Network  Problem; 

Consider  a  network  connecting  two  given  points  by  way  of 
a  number  of  intermediate  points,  where  each  link  of  the  network 
has  a  number  assigned  to  it  representing  its  capacity.  Assuming 
a  steady  state  condition,  find  a  maximal  flow  from  one  given 
point  to  the  other.  £l6j 

Some  problems  solved  by  dynamic  programming. 

Cargo  Loading  Problem; 

A  veesel  of  weight  capacity  x  is  to  be  loaded  with  an  assort¬ 
ment  of  items.  Each  item  has  a  weight  and  value  V^.  The  value 
may  be  a  non-linear  or  stochastic  function  of  number  loaded.  To 
determine  an  integral  number  of  eaoh  item  to  load  in  order  to 
maximise  the  value  of  cargo  eubject  to  the  weight  restriction.  |^4 

Kleeile  Allocation  Problem: 

M  missiles  are  to  be  allocated  to  N  targets.  Targets  are  of 
various  values  and  defense  levels.  Probability  of  a  missile  sur¬ 
viving  defenses  and  destroying  target  ie  an  exponential  function 
of  the  number  allocated.  Now  many  micelles  should  be  allocated 


P-«5 

6-22"56 

-L*- 


to  each  target  to  maxlmisa  total  expected  damage’  *1^ 

Optimal  Cl  lab  Problems 

What  path  should  an  airplane  follow  in  oruer  to  minimise 
tima-to-climb  from  takeoff  to  combat  conditions?  [I*] 

larly  Warning  Nadar  Not  Problem: 

Radar  protoctlon  circlss  an  constructed  around  oach  of  a 
complex  of  airbases.  Protoctlon  affordod  oach  bast  lo  a  function 
of  ths  radius  of  lto  clrclo.  A  sot  of  radii  lo  doslrod  which 
affords  tho  bass  complex  a  flxod  protoctlon  at  ml ml—  dofonso 
•ystom  perimeter.  [15] 

SlgopTalua  Calculation  Problem: 

Tho  eigenvalues  aosoclatod  with  tho  dlfforontlal  aquation 

> 

u"«X  0(t)u«O  art  obtaload.  Wo  aro  lntorootod  In  tho  values  of 
X*  which  yield  non-trivial  solutions  u*  QQ 

Conclusions 

Tho  potential  usor  of  olthor  of  those  programing  techniques 
should  first  consider  tho  factors  discussed  In  the  preceding  sections . 
If  the  problem  Is  essentially  linear  In  Its  assumptions,  there  le 
little  doubt  that  linear  programing  offers  the  superior  moans  of 
solution*  If  tho  pr 00000  is  of  a  multi-stags  nature  and  Its  true,  or 
reduced,  dimension  Is  not  largs,  dynamic  programing  should  bo  capable 
of  prodding  a  solution  unperturbed  by  non-linear,  stochastic,  or 


integral  restraints. 


It  ahould  be  noted,  how*r*r,  that  both  technique#,  whsn 
artfully  applied,  ha  ra  eolred  problem#  that  at  first  flanca  did 
not  fall  within  tha  limit ad  domain*  daacribad  abort. 

Finally,  it  i*  important  to  roalite  that  both  linaar  and  , 
dynamic  programing  roprasant  powerful  analytic  tool*,  Interesting 
theorems  hare  been  prored  and  atructura*  of  solution*  which  are  not 
computationally  obtainable  har*  bean  demons t rated  by  Mathematical 
application  of  tha  simplex  or  dual -simplex  algorithm*  of  linaar 
programing  and  functional  aquation  theory  of  dynamic  programing. 
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